Advanced topics

Unified exceptions

DataObjects.Net translates ADO.NET provider-specific exceptions to its own provider-independent exceptions. This should dramatically simplify any code relying on clasas on underlying exception, such as transactional reprocessing code.

Base class for any storage-level exception is named StorageException: It also serves as a base class for exceptions that are not related to SQL execution errors.

This class has many decendants. The most important are OperationTimeoutException, SyntaxErrorException, ReprocessableException, ConstraintViolationException.

Let’s look at them. The first is OperationTimeoutException. It’s thrown when current operation timed-out. This might happen if server became unavailable or internal timeout reached.

SyntaxErrorException is thrown when server didn’t accept the generated SQL query. Typically such error means that either RDBMS does not support particular feature or DataObjects.Net generated an invalid SQL.

ReprocessableException is a base class for exceptions related to transaction isolation errors. It has two decendants: DeadlockException and TransactionSerializationFailureException. First is thrown when dead-lock occured during execution and current transaction has been chosen as a victim to resolve dead-lcok. The second exception is thrown for other transaction isolation-related errors.

Finally, there is ConstraintViolationException. As the name suggests it wraps errors related to RDBMS constraints on data. It has three descendants for each constraint type: CheckConstraintViolationException, ReferentialConstraintViolationException, UniqueConstraintViolationException.

CheckConstraintViolationException is thrown when CHECK constraint is violated. It also thrown in the case of violation of NOT NULL constraint.

ReferentialConstraintViolationException is used for referential constraint errors. In other words, it’s thrown when FOREIGN KEY is not satisfied. This could mean inserting row that references non-existent row or removing row that is being referenced by other rows. DataObjects.Net handles references for you, so generally you should not get this exception.

UniqueConstraintViolationException wraps errors for PRIMARY KEY and UNIQUE constraints. Also it’s used for duplication errors with unique indexes.

Versions and concurrency control

DataObjects.Net supports both optimistic and pessimistic concurrency control:

  • Entity.Lock method and IQueryable<T>.Lock extension methods provide support for pessimistic concurrency control;
  • VersionInfo type and a set of supplementary classes (VersionSet, VersionValidator, VersionCapturer) provide support for optimistic concurrency control.

Pessimistic concurrency control: Lock method group

Lock methods put shared or exclusive lock on the specified rows in the database. Entity.Lock method puts lock on rows that correspond to Entity instance. Let’s take a look at quick example:

// Increments number of friends for the specified user
public void IncrementFriendsCountWithLock(User user)
{
    // Exclusively locks rows corresponding to the specified `User` entity.
    // If this object is already locked by other transaction an exception
    // would be thrown.
    user.Lock(LockMode.Exclusive, LockBehavior.ThrowIfLocked);

    // After lock has been obtained change to object
    user.FriendsCount++;
}

IQueryable<T>.Lock extension method puts lock on rows that correspond to the underlying query result. Here is example of its usage:

// Set IsArchied flag on all documents that were created earlier than
// specified date.
public void ArchiveOldDocuments(DateTime boundary)
{
    // Query the data and put exclusive lock on all obtained rows.
    var documents = session.Query.All<Document>
        .Where(doc => doc.Date < boundary)
        .Lock(LockMode.Exclusive, LockBehavior.ThrowIfLocked)
        .ToList();

    // Modify the document.
    foreach (var doc in documents)
        doc.IsArchived = true;
}

Lock() methods accept two arguments LockMode and LockBehavior. These control type of the lock to obtain and behavior in the case lock could not be obtained.

Locking is rather complex mechanism deeply integrated into almost any RDBMS to provide transaction isolation. If, after reading this part, you’ve got an imagination this is something simple, most likely you’re wrong. Transaction isloation-related concepts are frequently musinderstood. So if you feel you don’t fully udnerstand this (e.g. when locks are placed automatically, what types of locks are there, what is index range lock, what happens when you try to lock a resource that is already locked, what difference between locking and MVCC, etc.), we recommend you to read more about this. You can start e.g. from this article, and go further until you’ll be fluent with all the terms (you should already know the keywords).

Logging

Basics

DataObjects.Net’s logging infrastructure consists of 2 main components: loggers and log appenders.

Loggers

  • Orm - logs Session & Transaction-related events and exceptions.
  • Orm.Building - logs events during the Domain building process.
  • Storage.Providers.Sql - logs SQL statements sent to database server.
  • Orm.Upgrade - logs events during database schema upgrade.

Note

Loggers have such names by historical reasons, they might be changed in future versions.

Appenders

  • Console - redirects output to application’s console window, if any. Useful for small & test projects.
  • DebugOnlyConsole - the same as Console but appends log data only when a project is run in Debug mode.
  • File - appends log data to a specified file. If file is absent, it will be created. Useful for development & production environments.
  • Error - writes log info to standard error output stream.
  • Null - writes to /dev/null.

Event types

  • Info
  • Warning
  • Error
  • FatalError

Configuring log output

Configuration of built-in logging is made in application configuration file (app.config or web.config).

First of all, include Xtensive.Diagnostics section into configSections node:

<configSections>
  <section
    name="Xtensive.Diagnostics"
    type="Xtensive.Diagnostics.Configuration.ConfigurationSection, Xtensive.Core" />
    ...

Next step is to configure appenders:

<Xtensive.Diagnostics>
  <logs>
    <!-- Use these settings for Orm log -->
    <log name="Orm"
         events="Info, Warning, Error, FatalError"
         format="Release"
         provider="File"
         fileName="Orm.log" />
    <!-- Use these settings for Sql log -->
    <log name="Storage.Providers.Sql"
         events="Info, Warning, Error, FatalError"
         format="Release"
         provider="Console"/>
  </logs>
</Xtensive.Diagnostics>

Log formats

  • Default or Comprehensive - logs date-time information in relative format. Useful for performance measuring purposes mainly.
  • Release - logs date-time data in absolute format. Recommended for both development & production.

The example of log:

2012-12-31 00:00:02,052 DEBUG Storage.Providers.Sql - Session 'Default, #9'. Creating connection 'sqlserver://*****'.
2012-12-31 00:00:02,052 DEBUG Storage.Providers.Sql - Session 'Default, #9'. Opening connection 'sqlserver://*****'.
2012-12-31 00:00:02,052 DEBUG Storage.Providers.Sql - Session 'Default, #9'. Beginning transaction @ ReadCommitted.
2012-12-31 00:00:02,068 DEBUG Storage.Providers.Sql - Session 'Default, #9'. SQL batch: SELECT [a].[Id], [a].[TypeId], [a].[Name], [a].[Code], ...
2012-12-31 00:00:02,068 DEBUG Storage.Providers.Sql - Session 'Default, #9'. Commit transaction.
2012-12-31 00:00:02,068 DEBUG Storage.Providers.Sql - Session 'Default, #9'. Closing connection 'sqlserver://*****'.

Built-in services

Some very specific tasks can’t be easily done with the help of public API. For those cases DataObjects.Net provides a set of so-called “internal” services that usually can be obtained through Session.Services endpoint or through static helpers. The services’ API surface is not final and may be changed in the future versions.

Note

Use Xtensive.Orm.Services namespace to refer to the services.

SessionStateAccessor

Exposes methods to internal state of Session.

using Xtensive.Orm.Services;

using (var session = Domain.OpenSession()) {
  using (var tx = session.OpenTransaction()) {
    var sessionAccessor = DirectStateAccessor.Get(session);

    // Number of entities in the cache
    int count = sessionAccessor.Count;

    // List all entities in session cache
    foreach(var entities in sessionAccessor)
      // apply some action

    // Resolve entity from session cache by key
    var entity = sessionAccessor[myKey];

    // Invalidates state of session cache
    sessionAccessor.Invalidate();

    tx.Complete();
  }
}

PersistentStateAccessor

Exposes methods to access internal state of a Persistent object. PersistentFieldState can be of two kinds: Loaded or Modified.

using Xtensive.Orm.Services;

using (var session = Domain.OpenSession()) {
  using (var tx = session.OpenTransaction()) {

    var animal = session.Query.All<Animal>().First();
    animal.Name = "Tiger";

    var accessor = DirectStateAccessor.Get(animal);

    // Checking the state of field
    var state = accessor.GetFieldState("Name");
    // state is PersistentFieldState.Modified

    tx.Complete();
  }
}

EntitySetStateAccessor

Exposes methods to internal state of EntitySet.

using Xtensive.Orm.Services;

using (var session = Domain.OpenSession()) {
  using (var tx = session.OpenTransaction()) {

    var person = session.Query.All<Person>().First();

    var accessor = DirectStateAccessor.Get(person.Pets);

    // Number of keys cached
    long count = accessor.Count;

    // Check whether count is available without query to database
    accessor.IsCountAvailable;

    // Check whether EntitySet is fully loaded from database
    accessor.IsFullyLoaded;

    // Check whether a key is cached or not
    accessor.Contains(myKey);

    // Enumerates all cached keys
    foreach(var key in accessor)
      // apply some action

    tx.Complete();
  }
}

DirectEntityAccessor

Exposes methods for creating instances of Persistent types and accessing their persistent fields.

using Xtensive.Orm.Services;

using (var session = Domain.OpenSession()) {
  using (var tx = session.OpenTransaction()) {

    var accessor = session.Get<DirectEntityAccessor>();

    // Methods for creating entities
    accessor.CreateEntity(typeof(Animal));
    accessor.CreateEntity(typeof(Animal), tuple);
    accessor.CreateEntity(myKey);

    // Methods for creating structures
    accessor.CreateStructure(typeof(Address));
    accessor.CreateStructure(typeof(Address), tuple);

    Animal animal = session.Query.All<Animal>().First();

    // Methods for accessing value of persistent fields
    FieldInfo nameField = Domain.Model.Types[typeof(Animal)].Fields["Name"];
    accessor.GetFieldValue(animal, nameField);
    accessor.SetFieldValue(animal, nameField, "Tiger");

    // Methods for accessing value of reference fields without fetching referenced entity
    FieldInfo ownerField = Domain.Model.Types[typeof(Animal)].Fields["Owner];
    Key ownerKey = accessor.GetReferenceKey(animal, ownerField);
    accessor.SetReferenceKey(animal, ownerField, newKey);

    tx.Complete();
  }
}

DirectEntitySetAccessor

Exposes methods for manipulating fields of EntitySet<T> type in generic way.

using Xtensive.Orm.Services;

using (var session = Domain.OpenSession()) {
  using (var tx = session.OpenTransaction()) {

    var accessor = session.Get<DirectEntitySetAccessor>();

    Person person = session.Query.All<Person>().First();
    FieldInfo petsField = Domain.Model.Types[typeof(Person)].Fields["Pets"];

    // Accessing instance of EntitySet
    accessor.GetEntitySet(person, petsField);

    // Methods for manipulating content of EntitySet
    accessor.Add(person, petsField, myAnimal);
    accessor.Remove(person, petsField, myAnimal);
    accessor.Clear(person, petsField);

    tx.Complete();
  }
}

DirectSqlAccessor

Exposes methods to access low-level objects like connection, command, transaction.

using Xtensive.Orm.Services;

using (var session = domain.OpenSession()) {
  using (var tx = session.OpenTransaction()) {

    var accessor = session.Services.Demand<DirectSqlAccessor>();

    var command = accessor.CreateCommand();
    command.CommandText = "DELETE FROM [dbo].[Animal];";
    command.ExecuteNonQuery();

    // It is a good idea to invalidate session cache after any direct manipulation
    DirectStateAccessor.Get(session).Invalidate();

    t.Complete();
  }
}

QueryFormatter

Provides methods for formatting LINQ queries.

using Xtensive.Orm.Services;

using (var session = Domain.OpenSession()) {
  using (var tx = session.OpenTransaction()) {

    var query = session.Query.All<FakeClass>().Where(f => f.Id > 0);
    var formatter = session.Services.Get<QueryFormatter>();

    // Query is translated to SQL
    Console.WriteLine(formatter.ToSqlString(query));

    // Query is formatted in C# expression notation
    Console.WriteLine(formatter.ToString(query));

    // Creates DbCommand based on query
    var command = formatter.ToDbCommand(query);

    tx.Complete();
  }
}

Customizing LINQ translation

LINQ translator in DataObjects.Net handles wide variety of methods of base class library types, such as String.Substring():

var substrings = session.Query.All<Person>()
  .Select(p => p.Name.Substring(2, 3));

LINQ translation pipeline is not limited by this set of methods. There are two ways of extending it:

  • Implementing custom SQL compilers. Such compilers define translation of a particular method call, property or field access to SQL DOM expression.
  • Implementing custom LINQ expression rewriters. These allow rewriting expressions that use non-persistent properties, methods and fields to expressions that are known to DataObjects.Net LINQ translator.

First of all, you must choose members to create custom compilers for and declare compiler container type. Compiler container is a special static class exposing member compilers as its static methods. It must be marked with [CompilerContainer] attribute:

[CompilerContainer(typeof (Expression))]
public static class CustomLinqCompilerContainer
{
}

Dependently on required compiler type you should pass the following argument to [CompilerContainer] constructor:

  • typeof (Xtensive.Sql.Dml.SqlExpression) to write SQL compiler
  • typeof (System.Linq.Expressions.Expression) for LINQ expression rewriter

Now you can write your own member compilation methods (member compilers) inside it. To define a compilation method (compiler) you need to create a static method and mark it with [Compiler] attribute. Each member compiler matches a single member of a particular type. Each time this member is encountered in LINQ expression corresponding compiler is invoked to provide transformation. Member compiler should have a signature based on the following rules:

  • If target member is a generic method or target member belongs to a generic type MemberInfo parameter should be added. This parameter is used to pass actual member for which this compiler is invoked. It allows determining actual generic parameters which where used to construct a particular generic member.
  • If target member is an instance member the corresponding parameter should be added. This parameter defines expression representing instance which is used for accessing member.
  • If target member is a method or indexed property you should add as many parameters as target member has.
[Compiler(typeof(CustomSqlCompilerStringExtensions),
  "BuildAddressString",
  TargetKind.Method | TargetKind.Static)]
public static SqlExpression BuildAddressString(
  SqlExpression countryExpression,
  SqlExpression streetExpression,
  SqlExpression buildingExpression)

[Compiler] attribute parameters:

  • Type where member is defined.
  • Name of the member
  • Member kind: static or instance; method, property or field an so on.

You can (but not required to) apply [Type] attribute to any argument in a compiler method declaration. When [Type] attributes are provided, compiler resolver will consider argument types specified there during the matching process. Such declarations allow creating different compilers for different overloads of the same method. If there are no overloaded methods or overloading is based solely on number of parameters such declarations are not required.

Finally, you must register compiler container in the domain configuration:

var config = new DomainConfiguration("sqlserver://localhost/DO40-Tests");
config.Types.Register(typeof (CustomLinqCompilerContainer));

If all these steps are completed, DataObjects.Net will use your compiler container in the corresponding domain.

Customizing compilation to SQL

Result of compilation of any .NET expression to SQL is SqlExpression object from Xtensive.Sql.Dml namespace. SQL compilers are dealing with internal abstractions (Tuple field values), so there are no entities, structures and other high-level objects. The only types you can deal with are primitive types supported by DataObjects.Net, such as String and DateTime.

To illustrate the usage of custom SQL compiler, let’s imagine we want to extend string type with GetThirdChar() method returning 3rd character in the string.

First of all, we need this method itself:

public static class CustomCompilerStringExtensions
{
  public static char GetThirdChar(this string source)
  {
    return source[2];
  }
}

A query using it:

var thirdChars = session.Query.All<Person>().Select(p => p.Name.GetThirdChar());

Let’s write its compiler:

[CompilerContainer(typeof(SqlExpression))]
public static class CustomStringCompilerContainer
{
  [Compiler(typeof (CustomCompilerStringExtensions),
    "GetThirdChar", TargetKind.Method | TargetKind.Static)]
  public static SqlExpression GetThirdChar(SqlExpression _this)
  {
    return SqlDml.Substring(_this, 2, 1);
  }
}

If this compiler is registered in the domain, SQL DOM translator will convert any GetThirdChar() method call to SqlDml.Substring(_this, 2, 1) expression.

Let’s imagine another case: we want to build address string from its components:

public static string BuildAddressString(
    string country, string city, string building)
{
  return string.Format("{0}, {1}-{2}", country, city, building);
}

The compiler for this method:

[Compiler(typeof(CustomSqlCompilerStringExtensions),
  "BuildAddressString",
  TargetKind.Method | TargetKind.Static)]
public static SqlExpression BuildAddressString(
  SqlExpression countryExpression,
  SqlExpression streetExpression,
  SqlExpression buildingExpression)
{
  return SqlDml.Concat(
    countryExpression, SqlDml.Literal(", "),
    streetExpression, SqlDml.Literal("-"), buildingExpression);
}

Now this method can be used in LINQ queries:

var addresses = session.Query.All<Person>().Select(p =>
  CustomSqlCompilerStringExtensions.BuildAddressString(
  p.Address.Country, p.Address.City, p.Address.Building));

Finally, let’s try to create a compiler for built-in GetHashCode() method of string type. Since method is built-in, there should be just its compiler:

[Compiler(typeof(string), "GetHashCode", TargetKind.Method)]
public static SqlExpression GetHashCode(SqlExpression _this)
{
  // Return string length as its hash code
  return SqlDml.CharLength(_this);
}

Note that [Compiler] attribute uses typeof(string) as its first parameter, because GetHashCode() method we’re going to compile is overriden in string type.

This compiler makes possible to run the following query:

var hashCodes = session.Query.All<Person>()
  .OrderBy(p=>p.Id)
  .Select(p => p.Address.Country.GetHashCode());

Writing custom LINQ expression rewriter

LINQ expression rewriters provide much more convenient way of extending the LINQ translator:

  • SQL compilation is one of final steps in LINQ translation pipeline. So the abstractions you deal with (or return) here are SQL DOM primitives operating with primitive data types.
  • But LINQ rewriters operate on the first stage of translation, so all the expressions from original LINQ query are available there. Moreover, you can use (e.g. return) any expressions supported by DataObjects.Net LINQ translator here.

LINQ rewriters translate expressions to expressions, so you must apply [CompilerContainer(Expression)] attribute to a rewriter type and use Expression as common argument & result type in its methods.

[CompilerContainer(typeof (Expression))]
public static class CustomLinqCompilerContainer

Let’s extend persistent Person type with non-persistent FullName property:

public string FullName
{
  get { return string.Format("{0} {1}", FirstName, LastName); }
}

Its LINQ rewriter:

[Compiler(typeof (Person), "FullName", TargetKind.PropertyGet)]
public static Expression FullName(Expression personExpression)
{
  var spaceExpression = Expression.Constant(" ");
  var firstNameExpression = Expression.Property(personExpression, "FirstName");
  var lastNameExpression = Expression.Property(personExpression, "lastName");
  var methodInfo = typeof (string).GetMethod("Concat",
    new[] {typeof (string), typeof (string), typeof (string)});
  var concatExpression = Expression.Call(
    Expression.Constant(null, typeof(string)),
    methodInfo, firstNameExpression, spaceExpression, lastNameExpression);
  return concatExpression;
}

There is a way to reduce rewriter’s complexity – we can create LambdaExpression and bind its parameters:

[Compiler(typeof (Person), "FullName", TargetKind.PropertyGet)]
public static Expression FullName(Expression personExpression)
{
  // Since "ex" type is specified, C# compiler
  // allows to use Person properties:
  Expression<Func<Person, string>> ex =
    person => person.FirstName + " " + person.LastName;

  // Binding lambda parameters replaces parameter usage in lambda.
  // In this case resulting expression body looks like this:
  // personExpression.FirstName + " " + personExpression.LastName
  return ex.BindParameters(personExpression);
}

As you already know, we must register this rewriter:

var config = new DomainConfiguration("sqlserver://localhost/DO40-Tests");
config.Types.Register(typeof (CustomLinqCompilerContainer));

Now FullName property can be used in LINQ queries:

var fullNames = session.Query.All<Person>()
  .OrderBy(p => p.Id)
  .Select(p => p.FullName);

Now let’s imagine we need a method addding custom prefix to LastName property:

public string PrefixLastName(string prefix)
{
  return string.Format("{0}{1}", prefix, LastName);
}

Custom compiler for this method:

[Compiler(typeof (Person), "PrefixLastName", TargetKind.Method)]
public static Expression PrefixLastName(
  Expression personExpression, Expression prefixExpression)
{
  Expression<Func<Person, string, string>> ex
    = (person, prefix) => prefix + person.LastName;
  return ex.BindParameters(personExpression, prefixExpression);
}

Now it’s possible to use PrefixLastName method in LINQ queries:

var resultStrings = session.Query.All<Person>()
  .OrderBy(p => p.Id)
  .Select(p => p.PrefixLastName("Mr. "));
var resultStrings = session.Query.All<Person>()
  .OrderBy(p => p.Id)
  .Select(p => p.PrefixLastName(p.Id.ToString()));

All custom compilers must be deterministic: they should produce the same resulting expression for the same input. If this condition is violated, the query involving such a compiler will behave improperly while being used inside Session.Query.Execute() method. This does not mean the result of evaluation of returned expression must be deterministic as well. So you can use e.g. DateTime.Now (non-deterministic function) there – but you must ensure the expression you return is the same for the same input.